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(54) Intravascular ultrasound enhanced image and signal processing 

(57) A device and method for intravascular ultra- 
sound imaging. An ultrasound signal transmitter and 
detector is introduced into and may be moved through a 
bodily lumen. The ultrasound signal transmitter and 
detector transmits ultrasonic signals and detects 
reflected ultrasound signals which contain information 
relating to the bodily lumen. A processor coupled to the 
ultrasound signal transmitter and detector is pro- 
grammed to derive a first image or series of images and 
a second image or series of images from the detected 
ultrasound signals. The processor is also programmed 
to compare the second image or series of images to the 
first image or series of images respectively. The proces- 
sor may be programmed to stabilize the second image 
in relation to the first image and to limit drift. The proc- 
essor may also be programmed to monitor the first and 
second images for cardiovascular periodicity, image 
quality temporal change and vasomotion. It can also 
match the first series of images and the second series 
of images. 
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Description 

Field Of The Invention 

5 The present invention relates to a device and method for enhanced image and signal processing for Intravascular 
Ultrasound ("IVUS"), and more specifically, to a device and method for processing IVUS image and signal information 
which will enhance the quality and utility of IVUS images. 

Background Information 

10 

IVUS images are derived from a beam of ultrasonic energy projected by apparatus such as a transducer or trans- 
ducer array located around, along or at the tip of a catheter inserted within a blood vessel. An ultrasound beam from 
the apparatus is continuously rotated within the blood vessel forming a 360° internal cross sectional image, i.e., the 
image is formed in a transverse (X-Y) plane. Depending on the specific apparatus configuration, the image may be 
15 derived either from the same transverse plane of the apparatus or from a transverse plane found slightly forward (i.e. , 
distal) of the transverse plane of the apparatus. If the catheter is moved inside and along the blood vessel (i.e., along 
the Z-axis), images of various segments (series of consecutive cross sections) of the vessel may be formed and dis- 
played. 

IVUS may be used in ail types of blood vessels, including but not limited to arteries, veins and other peripheral ves- 

20 sels, and in all parts of a body. 

The ultrasonic signal that is received (detected) is originally an analog signal. This signal is processed using analog 
and digital methods so as to eventually form a set of vectors comprising digitized data. Each vector represents the ultra- 
sonic response of a different angular sector of the vessel, i.e., a section of the blood vessel. The number of data ele- 
ments in each vector (axial sampling resolution) and the number of vectors used to scan a complete cross section 

25 (lateral sampling resolution) of the vessel may vary depending on the type of system used. 

The digitized vectors may initially be placed into a two-dimensional array or matrix having Polar coordinates, i.e., 
A(r, 8). In this Polar matrix, for example, the X axis corresponds to the r coordinate and the Y axis corresponds to the 8 
coordinate. Each value of the matrix is a value (ranging from 0-255 if the system is 8 bit) representing the strength of 
the ultrasonic response at that location. 

30 This Polar matrix is not usually transferred to a display because the resultant image will not be easily interpreted by 
a physician. The information stored in the Polar matrix A(r, 8) usually undergoes several processing stages and is inter- 
polated into Cartesian coordinates, e.g., X and Y coordinates (A(X, Y)) that are more easily interpreted by a physician. 
Thus, the X and Y axis of matrix A(X, Y) will correspond to the Cartesian representation of the vessel's cross-section. 
The information in the Cartesian matrix possibly undergoes further processing and is eventually displayed for analysis 

35 by a physician. Images are acquired and displayed in a variable rate, depending on the system. Some systems can 
acquire and display images in video-display rate, e.g., up to about 30 images per second. 

IVUS examination of a segment of a bodily lumen, i.e., vessel is generally performed by situating the catheter distal 
(i.e., downstream) to the segment to be reviewed and then the catheter is pulled back (pullback) slowly along the bodily 
lumen (Z-axis) so that successive images that form the segment are continuously displayed. In many cases the catheter 

40 is connected to a mechanical pulling device which pulls the catheter at a constant speed (i.e. , a typical speed is approx- 
imately 0.5 - 1 mm/sec). 

In IVUS imaging systems today the technique described above for displaying an image of a cross section of a bodily 
lumen, e.g., blood vessel, is generally used. These systems are deficient, however, because they do not include any 
form of stabilization of the images to compensate for movements of the catheter and/or bodily lumen, e.g., blood vessel. 
- 45 It is well known that during IVUS imaging of a bodily lumen, there is always motion exhibited by the catheter and/or the 
bodily lumen. This motion might be exhibited in the transverse (X-Y) plane, along the vessel axis (Z axis) or a combina- 
tion of those movements. The imaging catheter can also be tilted in relation to the vessel so that the imaging plane is 
not perpendicular to the Z axis (This movement shall be termed as angulation). These movements are caused by, 
among other things, beating of the heart, blood and/or other fluid flow through the lumen, vasomotion, forces applied by 
so the physician, and other forces caused by the physiology of the patient. 

In IVUS systems today, when the imaging catheter is stationary or when performing slow manual or mechanical 
pullback, relative movement between the catheter and the lumen is the primary factor for the change in appearance 
between successive images, i.e., as seen on the display and/or on film or video. This change in appearance occurs 
because the rate of change of an image due to movements is much greater than the rate of change in the real morphol- 
55 ogy due to pullback. 

Stabilization occurs when the images include compensation for the relative movement between the catheter and 
the lumen in successive images. Because none of the IVUS systems used today perform stabilization, there is no com- 
pensation for or correction of relative movements between the catheter and the lumen. As a result, morphological fea- 
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tures are constantly moving or rotating, i.e. , on the display and/or film or video. This makes it difficult for the physician 
to accurately interpret morphology in an IVUS dynamic display. Furthermore, when non-stabilized IVUS images are fed 
as an input to a processing algorithm such as 3D reconstruction or different types of filter that process a set of succes- 
sive images, this can lead to degraded performance and misdiagnosis or inaccurate determinations. 

5 Current IVUS imaging apparatus or catheters may have occasional malfunctions of an electronic or mechanical ori- 
gin. This can cause displayed images to exhibit both recognized or unrecognized artifacts and obscure the real mor- 
phology. Currently there is no automatic methods to determine whether images posses these types of artifacts which 
hamper the analysis of the images of the vessel or bodily lumen. 

The behavior of cardio-vascular function is generally periodic. The detection of this periodicity and the ability to 

10 establish correlation between an image and the temporal phase in the cardiac cycle to which it belongs is referred to as 
cardiac gating. 

Currently, cardiac gating is performed by using an external signal, usually an ECG (Electro-Cardiogram). However, 
ECG gating requires both the acquisition of the ECG signal and its interleaving (or synchronization) with the IVUS 
image. This requires additional hardware/software. 

15 Morphological features in IVUS images of blood vessels can be broken into three general categories: the lumen, 
i.e.. the area through which the blood or other bodily fluid flows; the vessel layers; and the exterior, /.e., the tissue or 
morphology outside of the vessel. Blood in most IVUS films (images) is characterized by a rapidly changing speckular 
pattern. The exterior of the vessel also alternates with high temporal frequency. Currently, the temporal behavior of pix- 
els and their textural attributes are not monitored automatically. 

20 Vasomotion in the context of bodily lumens, e.g. , blood vessel, is defined as the change in the caliber of the lumen, 
e.g., vessel. This change can be brought about by natural circumstances or under induced conditions. Vasomotion can 
have a dynamic component, i.e., dynamic change of the lumen's dimensions, e.g., vessel's caliber (contraction and 
dilation) during the cardiovascular cycle, and a baseline static component, i.e. , a change in the baseline caliber of the 
lumen, e.g., vessel. 

25 Vasomotion can be expressed as quantitative physiological parameters indicating the ability of the lumen, e.g. , ves- 
sel to change its caliber under certain conditions. These types of parameters have current and possibly future medical 
and diagnostic importance in providing information regarding the state of the lumen, e.g., vessel and the effect of the 
therapy performed. 

IVUS can be used to monitor vasomotion because it provides an image of the lumen's baseline caliber and its 
30 dynamic changes. Additionally, IVUS can be used to monitor whether the vasomotion is global (uniform), i.e. , where the 
entire cross-section of the lumen contracts/dilates in the same magnitude and direction. IVUS can also be used to 
determine whether the vasomotion is non-uniform which leads to local changes in the caliber of the lumen, i.e. . different 
parts of the lumen cross-section behave differently. 

Currently, all types of vasomotion monitoring by IVUS are performed manually. This is tedious, time consuming, 
35 and prevents monitoring of the vasomotion in real time. 

Interpretation of IVUS images is achieved through analysis of the composition of the static images and monitoring 
their temporal behavior. Most IVUS images can be divided into three basic parts. The most inner section is the flow pas- 
sage of the lumen, i.e. , the cavity through which matter, i.e. , blood, flows. Around the flow passage is the actual vessel, 
which may include blood vessels and any other bodily vessels, which is composed of multiple layers of tissue (and 
40 plaque, if diseased). Outside the vessel other tissue which may belong to the surrounding morphology, for example, the 
heart in a coronary vessel image. 

When the IVUS film is viewed dynamically, i.e., in film format, the pixels corresponding to matter flowing through 
the vessel and to the morphology exterior to the vessel exhibit a different temporal behavior than the vessel itself. For 
example, in most IVUS films, blood flowing through the vessel is characterized by a frequently alternating speckular pat- 
45 tern. The morphology exterior to the vessel also exhibits frequent alternation. Currently the temporal behavior of pixels 
in dynamic IVUS images is not monitored automatically. 

In current IVUS displays, if designed into the system, high frequency temporal changes are suppressed by means 
such as averaging over a number of images. However, this sometimes fails to suppress the appearance of features with 
high amplitudes, i.e. , bright gray values, and it also has a blurring effect. 
so The size of the flow passage of the lumen is a very important diagnostic parameter. When required for diagnosis, 
it is manually determined by, for example, a physician. This is accomplished by drawing the contour of the flow passage 
borders superimposed on a static image, e.g., frozen on video or on a machine display. This method of manual extrac- 
tion is time consuming, inaccurate and subject to bias. 

Currently, there is commercial image processing software for the automatic extraction of the flow passage. How- 
55 ever, these are based on the gray value composition of static images and do not take into account the different temporal 
behavior exhibited by the material, e.g., blood flowing through the passage as opposed to the vessel layers. 

During treatment of vessels, it is oommon practice to repeat IVUS pullback examinations in the same vessel seg- 
ments. For example, a typical situation is first to review the segment in question, evaluate the disease (if any), remove 
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the IVUS catheter, consider therapy options, perform therapy, e.g., PTCA-"balloon" or starting, and then immediately 
thereafter reexamine the treated segment using IVUS in order to assess the results of the therapy. To properly evaluate 
the results and fully appreciate the effect of the therapy performed, it is desirable that the images of the pre-treated and 
post-treated segments, which reflect cross sections of the vessel lying at the same locations along the vessel's Z-axis 

5 {i.e. , corresponding segments), be compared. To accomplish this comparison it must be determined which locations in 
the films of the pre-treatment IVUS images and post-treatment IVUS images correspond to one another. This proce- 
dure, called matching (registration) allows an accurate comparison of pre- and post-treatment IVUS images. 

Currently, matching is usually performed by viewing the IVUS pullback films of pre- and post-treatment segments, 
one after the other or side by side by using identifiable anatomical landmarks to locate the sequences that correspond 

10 visually to one another. This method is extremely imprecise and difficult to achieve considering that the images are 
unstable and often rotate and/or move around on the display due to the absence of stabilization and because many of 
the anatomical landmarks found in the IVUS pullback film of the pre-treatment segment may be disturbed or changed 
as a result of the therapy performed on the vessel. Furthermore, the orientation and appearance of the vessel is likely 
to change as a result of a different orientations and relative positions of the IVUS catheter in relation to the vessel due 

15 to its removal and reinsertion after therapy is completed. The matching that is performed is manual and relies primarily 
on manual visual identification which can be extremely time consuming and inaccurate. 

Summary Of The invention 

20 The present invention solves the problems associated with IVUS imaging systems currently on the market and with 
the prior art by providing physicians with accurate IVUS images and image sequences of the morphology being 
assessed, thereby enabling more accurate diagnosis and evaluation. 

The present invention processes IVUS image and signal information to remove distortions and inaccuracies 
caused by various types of motion in both the catheter and the bodily lumen. This results in both enhanced quality and 

25 utility of the IVUS images. An advantage provided by the present invention is that individual IVUS images are stabilized 
with respect to prior image(s), thereby removing negative effects on any later processing of multiple images. If the 
movements in each image are of the transverse type, then it is possible for the motion to be completely compensated 
for in each acquired image. 

The present invention also allows volume reconstruction algorithms to accurately reproduce the morphology since 
30 movement of the bodily lumen is stabilized. The present invention is applicable to and useful in any type of system 
where there is a need to stabilize images (IVUS or other) because a probe {e.g., ultrasonic or other) moving through a 
lumen experiences relative motion {i.e., of the probe and/or of the lumen). 

The present invention provides for detection of an ultrasonic signal emitted by ultrasonic apparatus in a bodily 
lumen, conversion of the received analog signal into Polar coordinates (A(r, 6)), stabilization in the Polar field, convert- 
35 ing the stabilized Polar coordinates into Cartesian coordinates (A(X, Y)), stabilization in the Cartesian field and then 
transferring the stabilized image as Cartesian coordinates to a display. Stabilized images, either in Polar or Cartesian 
coordinates, may be further processed prior to display or they might not be displayed. Conversion into Cartesian coor- 
dinates and/or stabilization in the Cartesian field may be done at any point either before or after stabilization in the Polar 
field. Additionally, either of Polar or Cartesian stabilization may be omitted, depending on the detected shift in the image 
40 and/or other factors. Furthermore, additional forms of stabilization may be included or omitted depending on the 
detected shift and/or other factors. 

For example, stabilization of rigid motion may be introduced to compensate for rotational motion (angular) or global 
vasomotion (expansion or contraction in the r direction) in the Polar field and/or for Cartesian displacement (X and/or Y 
direction) in the Cartesian field. 

• 45 Transverse rigid motion between the representations of successive images is called a "shift, " i.e., a uniform motion 
of all morphological features in the plane of the image. To stabilize IVUS images, the first step that is performed is "shift 
evaluation and detection." This is where the shift (if any) between each pair of successive images is evaluated and 
detected. The system may utilize a processor to perform an operation on a pair of successive IVUS images to deter- 
mine whether there has been a shift between such images. The processor may utilize a single algorithm or may select 
so from a number of algorithms to be used in making this determination. 

The system utilizes the algorithm(s) to simulate a shift in an image and then compares this shifted image to its pred- 
ecessor image. The comparisons between images are known as closeness operations which may also be known in the 
prior art as matching. The system performs a single closeness operation for each shift. The results of the series of 
closeness operations is evaluated to determine the location (direction and magnitude) of the shifted image that bears 
55 the closest resemblance to the predecessor unshifted image. An image can of course be compared in the same manner 
to its successor image. After the actual shift is determined, the current image becomes the predecessor image, the next 
image becomes the current image and the above operation is repeated. 

Using shift evaluation and detection, the system determines the type of transverse shift, e.g., rotational, expansion, 
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contraction, displacement (Cartesian), etc., along with the direction and magnitude of the shift. The next step is "shift 
implementation." This is where the system performs an operation or a series of operations on successive IVUS images 
to stabilize each of the images with respect to its adjacent predecessor image. This stabilization utilizes one or multiple 
"reverse shifts" which are aimed at canceling the detected shift. The system may include an algorithm or may select 

5 from a number of algorithms to be used to implement each "reverse shift." The logic which decides upon what reverse 
shift will actually be implemented on an image, prior to its feeding to further processing or display, is referred to as „shrft 
logic". Once the IVUS images are stabilized for the desired types of detected motion, the system may then transfer the 
Cartesian (or Polar) image information lor further processing and finally for display where the results of stabilization may 
be viewed, for example, by a physician. Alternatively, stabilization can be invisible to the user in the sense that stabili- 

w zation can be used prior to some other processing steps, after which, resulted images are projected to the display in 
their original non-stabilized posture or orientation. 

It is possible that the transverse motion between images will not be rigid but rather of a local nature, i.e., different 
portions of the image will exhibit motion in different directions and magnitudes. In that case the stabilization methods 
described above or other types of methods can be implemented on a local basis to compensate for such motion. 

is The present invention provides for detection of the cardiac periodicity by using the information derived only from 
IVUS images without the need for an external signal such as the ECG. This process involves closeness operations 
which are also partly used in the stabilization process. One important function of detecting periodicity [i.e. , cardiac gat- 
ing), when the catheter is stationary or when performing controlled IVUS pullback, is that it allows the selection of 
images belonging to the same phase in successive cardiac cycles. Selecting images based on the cardiac gating will 

20 allow stabilization of all types of periodic motion (including transverse, Z-axis and angulations) in the sense that images 
are selected from the same phase in successive heart-beats. These IVUS images, for example, can be displayed and 
any gaps created between them may be compensated for by filling in and displaying interpolated images. The IVUS 
images selected by this operation can also be sent onward for further processing. 

The closeness operations used for periodicity detection can also be utilized for monitoring image quality and indi- 

25 cate artifacts associated with malfunction of the imaging and processing apparatus. 

Operations used for shift evaluation can automatically indicate vasomotion. This can serve the stabilization process 
as vasomotion causes successive images to differ because of change in the vessel's caliber. If images are stabilized 
for vasomotion, then this change is compensated for. Alternatively, the information regarding the change in caliber may 
be displayed since it might have physiological significance. Monitoring of vasomotion is accomplished by applying 

30 closeness operations to successive images using their Polar representations, i.e., A(r, 0). These operations can be 
applied between whole images or between corresponding individual Polar vectors (from successive images), depend- 
ing on the type of information desired. Since global vasomotion is expressed as a uniform change in the lumen's caliber 
it can be assessed by a closeness operation which takes into account the whole Polar image. In general, any operation 
suitable for global stabilization in the Polar representation can be used to assess global vasomotion. 

35 Under certain conditions during IVUS imaging there may be non-uniform vasomotion, i.e., movement only in cer- 
tain sections of the IVUS image corresponding to specific locations in the bodily lumen. This may occur, for example, 
where an artery has a buildup of plaque in a certain location, thereby allowing expansion or contraction of the artery 
only in areas free of the plaque buildup. When such movement is detected the system is able to divide the ultrasound 
signals representing cross sections of the bodily lumen into multiple segments which are then each processed individ- 

40 ually with respect to a corresponding segment in the adjacent image using certain algorithm(s). The resulting IVUS 
images may then be displayed. This form of stabilization may be used individually or in conjunction with the previously 
discussed stabilization techniques. Alternatively, the information regarding the local change in vessel caliber can be dis- 
played since it might have physiological significance. 

The temporal behavior of pixels and their textural attributes could serve for: enhancement of display; and automatic 

45 segmentation (lumen extraction). If monitored in a stabilized image environment then the performance of the display 
enhancement and segmentation processes may be improved. 

According to the present invention, the temporal behavior of IVUS images may be automatically monitored. The 
information extracted by such monitoring can be used to improve the accuracy of IVUS image interpretation. By filtering 
and suppressing the fast changing features such as the matter, e.g. , blood flowing through the vessel and the morphol- 

50 ogy exterior to the vessel as a result of their temporal behavior, human perception of the vessel on both static images 
and dynamic images, e.g. , images played in cine form, may be enhanced. 

Automatic segmentation, i.e. , identification of the vessel and the matter, e.g. , blood flowing through the vessel may 
be performed by using an algorithm which automatically identifies the matter, e.g. . blood based on the temporal behav- 
ior of textural attributes formed by its comprising pixels. The temporal behavior that is extracted from the images can be 

55 used for several purposes. For example, temporal filtering may be performed for image enhancement, and detection of 
the changes in pixel texture may be used for automatic identification of the lumen and its circumference. 

In all IVUS images, the catheter itself (and imaging apparatus) is best to be eliminated from the image prior to per- 
forming stabilization or for monitoring. Failure to eliminate the catheter might impair stabilization techniques and moni- 
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taring. Elimination of the catheter may be performed automatically since its dimensions are known. 

The present invention also provides for automatic identification (/.a, matching or registration) of corresponding 
frames of two different I VUS pullback films of the same segment of a vessel, e.g. , pre-treatment and post-treatment. To 
compare a first I VUS pullback film, i.e. . a first IVUS imaging sequence, with a second IVUS pullback film, i.e., a second 
5 IVUS imaging sequence, of the same segment of a bodily lumen, for example, captured on video, film or in digitized 
form, the imaging sequences must be synchronized. Matching, which will achieve this synchronization, involves per- 
forming closeness operations between groups of consecutive images belonging to the two sets of IVUS imaging 
sequences. 

Out of one imaging sequence a group of consecutive images, termed the reference group, is selected. This group 
10 should be selected from a portion of the vessel displayed in both imaging sequences and it should be a portion on which 
therapy will not be performed since the morphology of the vessel is likely to change due to therapy. Another condition 
for this matching process is that the two imaging sequences are acquired at a known, constant and preferably the same 
pullback rate. 

Closeness operations are performed between the images of the reference group and the images from the second 
15 group which has the same number of successive images extracted from the second imaging sequence. This second 
group of images is then shifted by a single frame with respect to the reference group and the closeness operations are 
repeated. This may be repeated for a predetermined number of times and the closeness results of each frame shift are 
compared to determine maximal closeness. Maximal closeness will determine the frame displacement between the 
images of the two imaging sequences. This displacement can be reversed in the first or second film so that correspond- 
so ing images may be automatically identified and/or viewed simultaneously. 

Thus, corresponding images may be viewed, for example, to determine the effectiveness of any therapy performed 
or a change in the morphology over time. Additionally, the various types of stabilization discussed above may be imple- 
mented within or between the images in the two sequences, either before, during or after this matching operation. Thus, 
the two films can be displayed not only in a synchronized fashion, but also in the same orientation and posture with 
25 respect to one another. 

Brief Description Of The Drawings 

Figures 1 (a) and (b) show a two-dimensional array or matrix of an image arranged in digitized vectors in Polar and 
30 Cartesian coordinates, respectively. 

Figure 2 illustrates the results of a shift evaluation between two successive images in Cartesian coordinates. 

Figure 3 shows images illustrating the occurrence of drift phenomena in Polar and Cartesian coordinates. 

Figure 4 illustrates the effect of performing stabilization operations (rotational and Cartesian shifts) on an image. 

Figure 5 illustrates global contraction or dilation of a bodily lumen expressed in the Polar representation of the 
35 image and in the Cartesian representation of the image. 

Figure 6 shows an image divided into four sections for processing according to the present invention. 

Figure 7 shows a vessel, in both Cartesian and Polar coordinates, in which local vasomotion has been detected. 

Figure 8 illustrates the results of local vasomotion monitoring in a real coronary vessel in graphical form. 

Figure 9 shows an ECG and cross-correlation coefficient plotted graphically in synchronous fashion. 
40 Figure 10 shows a table of a group of cross-correlation coefficient values (middle row) belonging to successive 
images (numbers 1 through 10 shown in the top row) and the results of internal cross-correlations (bottom row). 

Figure 1 1 shows a plot of a cross-correlation coefficient indicating an artifact in IVUS images. 

Figure 12 shows an IVUS images divided into three basic parts: the lumen through which fluid flows; the actual ves- 
sel; and the surrounding tissue. 
- 45 Figure 1 3 illustrates the results of temporal filtering. 

Figure 14 shows an image of the results of the algorithm for automatic extraction of the lumen. 

Figure 15 illustrates the time sequence of a first film (left column), reference segment from the second film (middle 
column) and the images from the first film which correspond (or match) the images of the reference segment (right col- 
umn). 

50 

Detailed Description 

In intravascular ultrasound (IVUS) imaging systems the ultrasonic signals are emitted and received by the ultra- 
sonic apparatus, for example, a transducer or transducer array, processed and eventually arranged as vectors compris- 
55 ing digitized data. Each vector represents the ultrasonic response of a different angular sector of the bodily lumen. The 
number of data elements in each vector (axial sampling resolution) and the number of vectors used to scan the com- 
plete cross-section (lateral sampling resolution) of the bodily lumen depends on the specific IVUS system used. 

The digitized vectors are initially packed into a two-dimensional array or matrix which is illustrated in Figure 1(a). 
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Generally, this matrix has what are known as Polar coordinates, i.e. , coordinates A(r. 6). The X-axis of the matrix shown 
in Figure 1 (a) corresponds to the r coordinate while the Y-axis of the matrix corresponds to the r coordinate while the Y- 
axis of the matrix corresponds to the 0 coordinate. Each value of the matrix is generally a gray value, for example, rang- 
ing from 0-255 if it is 8 bit. representing the strength of the ultrasonic signal at that corresponding location in the bodily 

5 lumen. This Polar matrix may then be converted into a Cartesian matrix as shown in Figure 1 (b) having an X-axis and 
Y-axis which correspond to the Cartesian representation of the vessel's cross-section. This image may then be further 
processed and transferred to a display. The initial array and the display may each utilize either Polar or Cartesian coor- 
dinates. The values for the matrix may be other than gray values, for example, they may be color values or other values 
and may be less than or more than 8 bits. 

10 During an IVUS imaging pullback procedure the bodily lumen, hereinafter referred to as a vessel, and/or the imag- 
ing catheter may experience several modes of relative motion. These types of motion include: (1) Rotation in the plane 
of the image, i.e., a shift in the 6 coordinate of the Polar image; (2) Cartesian displacement, i.e., a shift in the X and/or 
Y coordinate in the Cartesian image; (3) Global vasomotion, characterized by a radial contraction and expansion of the 
entire vessel, i.e., a uniform shift in the r-coordinate of the Polar image; (4) Local vasomotion, characterized by a radial 

15 contraction and expansion of different parts of the vessel with different magnitudes and directions, i.e. , local shifts in the 
r-coordinate of the Polar image; (5) Local motion, characterized by different tissue motion which vary depending on the 
exact location within the image; and (6) Through plane motion, i.e., movements which are perpendicular or near per- 
pendicular (angulation) to the plane of the image. 

Stabilization of successive raw images is applicable to the first 5 types of motion described above because motion 

20 is confined to the transverse plane. These types of motion can be compensated for, and stabilization achieved, by trans- 
forming each current image so that its resemblance to its predecessor image is maximized. The first 3 types of motion 
can be stabilized using closeness operations which compare whole or large parts of the images one to another. This is 
because the motion is global or rigid in its nature. The 4th and 5th types of motion are stabilized by applying closeness 
operations on a localized basis because different parts of the image exhibit different motion. The 6th type of motion can 

25 be only partly stabilized by applying closeness operations on a localized basis. This is because the motion is not con- 
fined to the transverse plane. This type of motion can be stabilized using cardiovascular periodicity detection. 

The next sections shall describe methods for global stabilization, followed by a description of methods for local sta- 
bilization. Stabilization using cardiovascular periodicity detection shall be described in the sections discussing periodic- 
ity detection. 

30 To achieve global stabilization, shift evaluation is performed using some type of closeness operation. The close- 
ness operation measures the similarity between two images. Shift evaluation is accomplished by transforming a first 
image and measuring its closeness, i.e., similarity, to its predecessor second image. The transformation may be 
accomplished, for example, by shifting the entire first image along an axis or a combination of axes (X and/or Y in Car- 
tesian coordinates or r and/or 6 in Polar coordinates) by a single pixel (or more). Once the transformation, i.e., shift is 

35 completed the transformed first image is compared to the predecessor second image using a predefined function. This 
transformation is repeated, each time by shifting the first image an additional pixel (or more) along the same and/or 
other axis and comparing the transformed first image to the predecessor second image using a predefined function. 
After all of the shifts are evaluated, the location of the global extremum of the comparisons using the predefined function 
will indicate the direction and magnitude of the movement between the first image and its predecessor second image. 

40 For example, Figure 2 illustrates the results of a shift evaluation between two successive images in Cartesian coor- 
dinates. Image A is a predecessor image showing a pattern, e.g., a cross-section of a vessel, whose center is situated 
in the bottom right quadrant of the matrix. Image B is a current image showing the same pattern but moved in an upward 
and left direction and situated in the upper left quadrant of the matrix. The magnitude and direction of the movement of 
the vessel's center is indicated by the arrow. The bottom matrix is the C(shiftX, shiftY) matrix which is the resulting 

45 matrix after performing shift evaluations using some type of closeness operation. 

There are many different algorithms or mathematical functions that can be used to perform the closeness opera- 
tions. One of these is cross-correlation, possibly using Fourier transform. This is where the current and predecessor 
images each consisting of, for example, 256 x 256 pixels, are each Fourier transformed using the FFT algorithm. The 
conjugate of the FFT of the current image is multiplied with the FFT of the predecessor image. The result is inversely 

so Fourier transformed using the IFFT algorithm. The formula for cross-correlation using Fourier transform can be shown 
as follows: 

C = real(ifft2((fft2(A)) * conj(fft2(B)))) 

55 where: 

A = predecessor image matrix (e.g., 256 x 256); 
B « current image matrix (e.g., 256 x 256); 
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fft2 = two dimensional FFT; 

ifft2 = two dimensional inverse FFT; 

conj o conjugate; 

real = the real part of the complex expression; 
5 * = multiplication of element by element; and 
C = cross-correlation matrix. 

Evaluating closeness using cross-correlation implemented by Fourier transform is actually an approximation. This 
is because the mathematical formula for the Fourier transform relates to infinite or periodic functions or matrices, while 

10 in real life the matrices (or images) are of a finite size and not necessarily periodic. When implementing cross-correla- 
tion using FFT, the method assumes periodicity in both axes. 

As a result, this formula is a good approximation and it reflects the actual situation in the e-axis of the Polar repre- 
sentation of the image, however, it does not reflect the actual situation in the r-axis of the Polar representation or of the 
X- or Y-axis of the Cartesian representation of the image. 

15 There are a number of advantages to cross-correlation utilizing FFT. First, all values of the cross-oorrelation matrix 
C(shiftX, shiftY) are calculated by this basic operation. Furthermore, there is dedicated hardware for the efficient imple- 
mentation of the FFT operation, i.e. Fourier transform chips or DSP boards. 

Another algorithm that can be used to perform closeness operations is direct cross-correlation, either normalized 
or not. This is achieved by multiplying each pixel in the current shifted image by its corresponding pixel in the predeces- 

20 sor image and summing up all of the results and normalizing in the case of normalized cross-correlation. Each shift 
results in a sum and the actual shift will be indicated by the largest sum out of the evaluated shifts. The formula for 
cross-correlation can be shown by the following formula: 

25 B{x ' sNnX> /-BhffiY) * A(x t y) 

C(shiftX, shiftY) = x 



The formula for normalized cross correlation is 

30 

C(shiftX, shiftY) = £ B(x-shiftX, y-shift Y) * A{x. y) / 
x,y 

35 V£ (B{x-shiftX t y-shiftY) * B{x-shiftX, y-shiftY)) V£ (A(x, y) * A(x, y)) 

x,y x,y 

where: 

40 A = predecessor image matrix; 
B = current image matrix; 
* = multiplication of pixel by corresponding pixel; 
I = sum of all pixels in matrix; 
C = matrix holding results for all performed shifts. 

• 45 

Using this direct method of cross-correlation, C(shrftX, shiftY) can be evaluated for all possible values of shiftX and 
shiftY For example, if the original matrices, A and B, have 256 x 256 pixels each, then shiftX and shiftY values, each 
ranging from -127 to +128 would have to be evaluated, making a total of 256 x 256 = 65,536 shift evaluations in order 
for Cfsh'rftX, shiftY) to be calculated for all possible values of shiftX and shiftY Upon completion of these evaluations the 
so global maximum of the matrix is determined. 

Direct cross-correlation can be implemented more efficiently by lowering the number of required arithmetic opera- 
tions. In order to detect the actual shift between images, evaluation of every possible shiftX and shiftY is not necessary. 
It is sufficient to find the location of the largest C(shiftX ( shiftY) of all possible shiftX and shiftY. 

A third algorithm that can be used to perform closeness operations is the sum of absolute differences (SAD). This 
55 is achieved by subtracting each pixel in one image from its corresponding pixel in the other image, taking their absolute 
values and summing up all of the results. Each shift will result in a sum and the actual shift will be indicated by the lowest 
sum. The formula for sum of absolute differences (SAD) can be shown as follows: 
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SAD = absolute(A - B) 
This formula can also be shown as follows: 

5 

£ abs(B(x - sNflX , y - ahlttY) -A(x. /)) 

C(shiftX, shrftY) = " 



ro where: 

A = predecessor image matrix; 
B = current image matrix; 
abs = absolute value. 
is - = subtraction of element by element; and 
I = sum of all differences. 

While the accuracy of each of these algorithms/formulas may vary slightly depending on the specific type of motion 
encountered and system settings, it is to be understood that no single formula can, a-priori be classified as providing 
20 the best or most accurate results. Additionally, there are numerous variations on the formulas described above and 
other algorithms/formulas that may be utilized for performing shift evaluation and which may be substituted for the algo- 
rithms/formulas described above. These algorithms/formulas also include those operations known in the prior art for 
use as matching operations 

Referring again to Figure 2, if the closeness operation performed is cross-correlation, then C(shiftX, shiftY) is called 
25 the cross-correlation matrix and its global maximum (indicated by the black dot in the upper left quadrant) will be located 
at a distance and direction from the center of the matrix (arrow in matrix C) which is the same as that of the center of 
the vessel in Image B relative to the center of the vessel in image A (arrow in Image B). 

If the closeness operation performed is SAD, then the black dot would indicate the global minimum which will be 
located at a distance and direction from the center of the matrix (arrow in matrix C) which is the same as that of the 
30 center of the vessel in Image B relative to the center of the vessel in Image A (arrow in Image B). 

Rotational motion is expressed as a shift along the current Polar image in the O-coordinate relative to its predeces- 
sor. The rotational shift in a current image is detected by maximizing the closeness between the current Polar image 
and its predecessor. Maximum closeness will be obtained when the current image is reversibly shifted by the exact 
magnitude of the actual shift. In for example, a 256 x 256 pixel image, the value of the difference (in pixels) between 128 
35 and the 6-coordinate of the maximum in the cross-correlation image (minimum in the SAD image), will indicate the 
direction (positive or negative) and the magnitude of the rotation. 

Global vasomotion is characterized by expansion and contraction of the entire cross section of the vessel. In the 
Polar image this type of motion is expressed as movement inwards and outwards of the vessel along the r-axis. Vaso- 
motion can be compensated by performing the opposite vasomotion action on a current Polar image in relation to its 
40 predecessor Polar image using one of the formulas discussed above or some other formula. In contrast to angular sta- 
bilization, vasomotion stabilization does not change the orientation of the image but actually transforms the image by 
stretching or compressing it. 

Cartesian displacement is expressed as a shift in the X-axis and/or Y-axis in the Cartesian image relative to its 
predecessor. This type of motion is eliminated by shifting the Cartesian image in an opposite direction to the actual shift. 

45 Thus. Cartesian displacement, in the Cartesian representation, can be achieved by essentially the same arithmetic 
operations used for rotational and vasomotion stabilization in the Polar representation. 

The number of shift evaluations necessary to locate the global extremum (maximum or minimum, depending on the 
closeness function) of C(shiftX, shiftY) may be reduced using various computational techniques. One technique, for 
example, takes advantage of the fact that motion between successive IVUS images is, in general, relatively low in rela- 

so tion to the full dimensions of the Polar and/or Cartesian matrices. This means that CfshiftX, shiftY) can be evaluated 
only in a relatively small portion around the center of the matrix, i.e. , around shiftX = 0, shiftY = 0. The extremum of that 
portion is assured to be the global extremum of matrix C(shjftX, shiftY) including for larger values of shiftX and shiftY. 
The size of the minimal portion which will assure that the extremum detected within it is indeed a global extremum var- 
ies depending on the system settings. The number of necessary evaluation operations may be further reduced by rely- 

55 ing on the smoothness and monotonous property expected from the C matrix (especially in the neighborhood of the 
global extremum). Therefore, if the value in the C(shiftX, shiftY) matrix at a certain location is a local extremum (e.g. , in 
a 5 x 5 pixel neighborhood), then it is probably the global extremum of ail of matrix C(shiftX, shiftY). 

Implementing this reduction of the number of necessary evaluations can be accomplished by first searching from 
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the center of the matrix (shiftX = 0, shiftY = 0) and checking a small neighborhood, e.g. , 5x5 pixels around the center. 
If the local extremum is found inside this neighborhood then it is probably the global extremum of the whole matrix 
C(shiftX, shiftY) and the search may be terminated. If, however, the local extremum is found on the edges of this neigh- 
borhood, e.g., shiftX = -2, shiftX = 2, shiftY = -2 or shiftY = 2, then the search is repeated around this pixel until a 
5 C(shiftX, shiftY) value is found that is bigger (smaller) than all of its close neighbors. 

Because in a large number of images there is no inter-image motion, the number of evaluations needed to locate 
the global extremum in those cases, will be approximately 5 x 5 ■ 25, instead of the original 65,536 evaluations. 

The number of necessary evaluation operations may also be reduced by sampling the images. For example, rf 256 
x 256 sized images are sampled for every second pixel then they are reduced to 128 x 128 sized matrixes. In this case, 
w direct cross-correlation or SAD, between such matrixes involve 128 x 128 operations instead of 256 x 256 operations, 
each time the images are shifted one in relation to the other. Sampling, as a reduction method for shift evaluation oper- 
ations can be interleaved with other above described methods for reduction. 

Referring again to Figure 2, as a result of the closeness operation, the indicated shiftX will have a positive value 
and shiftY a negative value. In order to stabilize Image 8, /.e. ( compensate for the shifts in the X and Y directions, shift 
is logic will reverse the shifts, i.e. , change their sign but not their magnitude, and implement these shifts on the matrix cor- 
responding to Image B. This will artificially reverse the shift in Image B and cause Image B to be unshifted with respect 
to Image A. 

The actual values used in the closeness calculations need not necessarily be the original values of the matrix as 
supplied by the imaging system. For example, improved results may be achieved when the original values are raised to 
20 the power of 2, 3 or 4 or processed by some other method. 

The imaging catheter and the enclosing sheath appear as constant artifacts in all IVUS images. This feature 
obscures closeness operations performed between images since it is not part of the morphology of the vessel. It is, 
therefore, necessary to eliminate the catheter and associated objects from each image prior to performing closeness 
operations, i.e., its pixels are assigned a value of zero. The elimination of these objects from the image may be per- 
25 formed automatically since the catheter's dimensions are known. 

Shift evaluation and implementation may be modular. Thus, shift evaluation and implementation may be limited to 
either Polar coordinates or Cartesian coordinates individually, or shift evaluation and implementation may be imple- 
mented sequentially for Polar and Cartesian coordinates. Presently, because imaging in IVUS systems is generally 
organized by first utilizing Polar coordinates and then converting into Cartesian coordinates, it is most convenient to per- 
30 form shift evaluation and implementation in the same sequence. However, the sequence may be modified or changed 
without any negative effects or results. 

The shift evaluation process can be performed along one or two axis. In general, two dimensional shift evaluation 
is preferred even when motion is directed along one axis. Shift implementation may be limited to both axis, one axis or 
neither axis. 

35 There is not a necessary identity between the area in the image used for shift evaluation and between the area on 
which shift implementation is performed. For example, shift evaluation may be performed using a relatively small area 
in the image while shift implementation will shift the whole image according to the shift indicated by this area. 

A trivial shift logic is one in which the shift implemented on each image (thereby forming a stabilized image) has a 
magnitude equal, and in opposite direction, to the evaluated shift. However, such logic can result in a process defined 

40 as Drift. Drift is a process in which implemented shifts accumulate and produce a growing shift whose dimensions are 
significant in relation to the entire image or display. Drift may be a result of inaccurate shift evaluation or non-transverse 
inter-image motion at some part of the cardiovascular cycle. When Cartesian stabilization is implemented, drift can 
cause, for example, the shifting of a relatively large part of the image out of the display. When rotational stabilization is 
implemented, drift can cause the increasing rotation of the image in a certain direction. 
f 45 Figure 3 is an image illustrating the occurrence of drift in Polar and Cartesian coordinates. The left image is the 
original display of the image while the right image is the same image after Polar and Cartesian stabilization has been 
performed. Note how the right image is rotated counter-clockwise in a large angle and shifted downward in relation to 
the left image. In this case, rotational and Cartesian shift implementation do not compensate for actual shifts in the 
image, but rather arise from inaccurate shift evaluation. 

so The shift logic must be able to deal with this drift so that there will be a minimal implementation of mistaken evalu- 
ated shifts. One method for preventing, or at least limiting drift is by setting a limit to the magnitude of allowable shifts. 
This will minimize the drift but at the cost of not compensating for some actual shift. Additional methods can be used to 
prevent or minimize shift. These may possibly be interleaved with cardio-vascular periodicity detection methods dis- 
cussed later. 

55 The images shown in Figure 4 illustrate the effect of performing stabilization operations (rotational and Cartesian 
shifts) on an image. The left image is an IVUS image from a coronary artery as it would look on a large portion of a 
regular display (with catheter deleted) while the right image shows how the left image would be displayed after stabili- 
zation operations are implemented. 
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